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Introduction 

The  scope  of  this  feasibility  study  was  to  find  out  how  the  unique  properties  of  the 
Cellular  Neural  Network  (CNN)  focal-plane-array  processor  chip,  which  uses  analogic  algo¬ 
rithm  for  following  continuously  moving  objects,  could  improve  the  performance  of  PAL- 
based  vision  systems  described  in  the  project  SPC-994065  (Contract  Order  No.  F61775-99- 
WE065).  Therefore,  OPTOPAL  started  collaboration  with  AnaLogic  Computers  Ltd.,  who 
were  on  the  way  of  developing  special  algorithms  for  CNN-UM  focal  plane  array  processors. 

Being  aware  of  the  fact  that  the  currently  existing  64  by  64  CNN-UM  focal  plane  ar¬ 
ray  processor  chip,  -  and  even  the  128  by  128  version  under  construction,  -  cannot  handle 
moderately  complex  images  with  high  spatial  frequency,  and  the  processing  results  will  be¬ 
come  poor  if  a  low-resolution  chip  is  used,  we  planned  to  develop  a  system  where  we  get  two 
images  at  the  same  time:  one  from  the  existing  64  by  64  CNN-UM  focal  plane  array  processor 
chip,  while  another  from  a  camera  which  provides  so-called  Humanoid  PAL  images  of  high 
resolution.  This  means,  it  renders  a  "360°  peripheral  view"  and,  in  addition,  a  "foveal  view  of 
"about  20°.  Comparing  the  data  delivered  by  a  CNN-UM  focal  plane  array  and  those  pre¬ 
sented  by  the  Humanoid  PAL,  the  performance  of  some  military  video  systems  such  as 

1.  azimuth  determination  system  (ADS) 

2.  sniper  fire  location  display  (SFLD) 

3.  helicopter  blade  tracking  system  (HBTS) 
could  be  improved. 

The  research  was  conducted  on  various  levels: 

1.  A  Humanoid  PAL  with  increased  peripheral  viewing  angle  was  designed,  resulting 
in  workshop  drawings  suitable  for  manufacturing. 

2.  A  foveal  lens  with  viewing  angle  of  about  20°  was  designed,  resulting  in  workshop 
drawings  suitable  for  manufacturing. 

3.  A  mount  system  was  designed  which  allows  the  use  of  off-the-shelf  lens  systems  as 
relay  lenses,  resulting  in  workshop  drawings  suitable  for  manufacturing. 

4.  A  software  for  color  images  was  developed,  which  allows  a  quasi-real-time  proc¬ 
essing  speed,  about  15  images  per  second,  capable  to  unwrap  the  panoramic  ring-shaped  im¬ 
age,  and  it  can  also  perform  various  filtering  procedures,  fine  edge  detection,  major  edge  ex¬ 
traction,  summation,  subtraction,  etc.;  further,  it  can  also  handle  the  foveal  view,  including 
creation  of  histograms  in  any  selected  direction. 
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Increasing  the  viewing  angle  of  PAL 


PAL,  as  an  imaging  block  is  a  single  glass  piece  with  two  refractive  and  two  reflective 
surfaces,  each  of  which  could  be  plane,  concave  or 
convex.  Therefore,  the  possible  shapes  of  such  an 
optical  element  amount  to  the  number  of  iterative 
variations  of  the  fourth  class  that  can  be  formed  of 
three  elements,  i.e.,  to  81.  Fig.  1  shows  five  forms 
that  are  all  suitable  of  forming  360°  panoramic 
images,  however,  not  necessarily  of  the  same 
optical  quality. 

Until  recently,  our  PAL  designs  started  from 
the  shape  seen  in  the  middle  of  the  first  row,  mainly 
because  its  plane  surface.  It  helps  namely  a  lot  in 
designing  appropriate  mounts  for  a  given 
application  task. 


NOT  SCALED 


Fig.  1  Shape  variations  for  PAL 


Because  of  this  shape  a  virtual  360°  panoramic  image  -  better  to  say,  a  panoramic  image  vol¬ 
ume  -  of  the  3D  surroundings  is  formed  inside  the  glass  block,  which  represents  the  2D 
skeleton  of  the  3D  space. 

The  viewing  angle  of  the  PALs  with  the  mentioned  shape  had  approximately  a  view¬ 
ing  angle  of  about  40°  above  and  15°  below  the  optical  horizon  of  the  PAL.  (Optical  horizon 
(OH)  is  the  plane,  where  the  curved  surfaces  intersect,  it  is  perpendicular  to  the  optical  axis.) 

Shortly  after  we  began  to  design  a  new  PAL  with  a  total  viewing  angle  of  around  80° 
and  suitable  for  foveal  vision,  it  turned  out  that  the  optical  design  program  ZEMAX  EE  we 
intended  to  use  was  not  capable  for  optimization.  The  reason  was  that  ZEMAX-EE  and  all 
present  optical  design  programs  are  based  on  see-through-window  (STW)  principle.  This 
means  that  they  operate  in  Cartesian  coordinates,  and  not  in  polar  coordinates,  thus,  they  do 
not  "understand"  centric-minded  imaging  (CMI). 

A  common  feature  of  the  axis-symmetrical  imaging  optics  is  that  the  entire  optic  is  in¬ 
volved  in  imaging  the  object  points.  With  other  words,  all  optical  rays  issuing  from  an  object 
point  pass  through  the  optic  and  meet  in  a  single  point  at  the  image  plane.  This  is  the  reason 
why  an  entire  image  of  the  object  is  formed  even  when  a  portion  of  the  optic  is  blocked;  the 
speed  of  the  lens,  however,  may  decrease  and/or  the  image  quality  may  change.  Thus,  optimi¬ 
zation  of  such  a  so-called  see-through-window  (STW)  imaging  system  means  that  each  object 
point  when  imaged  should  provide  the  possibly  smallest  spot  at  the  image  plane,  the  limit  of 
which  is  defined  by  diffraction.  Therefore,  high  quality  optics  are  diffraction  limited  at  a 
given  wavelength  interval  and  viewing  angle.  Appropriate  variation  of  the  radii,  the  distances 
between  optical  elements,  and  the  refractive  index  of  the  optical  material  achieve  this,  so  that 
all  image  points  will  produce  a  spot  in  the  image  plane,  the  diameter  of  which  is  smaller  than 
the  Airy-diameter.  In  Fig.  2,  one  can  see  the  ray  tracing  of  two  triplets  of  the  same  focal 
length:  in  the  first  row  without  optimization,  in  the  second  row,  diffraction  limited. 

The  green  column  shows  the  distance  of  the  image  points  from  the  image  plane,  the 
blue  column  the  diameter  of  the  produced  spot,  and  the  percentage  of  the  spot  diameter  with 
respect  to  the  Airy-diameter.  It  well  demonstrates  that  optimization  of  conventional  STW  im¬ 
aging  optics  is  not  a  simple  task  since  too  many  variables  have  to  be  considered;  in  the  case  of 
triplets,  three  glass  materials,  six  radii,  and  five  distances. 
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Fig.  2  Ray  trcing  of  two  triplets  of  the  same  focal  length:  non-optimized  (upper  row),  diffraction-limited 

((bottom  row) 

In  designing  a  PAL,  a  single  centric  minded  imaging  block,  we  are  faced  with  the 
problem  that  optical  definitions  used  in  traditional  STW  imaging  loose  their  meaning  and 
validity.  Further,  only  a  portion  of  the  rays  emerging  from  the  object  points  in  space  and 
reaching  the  optical  surface  are  taking  part  in  image  forming. 

PAL  can  be  thought  to  be  built  up  from  several  small  units,  each  of  which  produce, 
from  a  part  of  the  3D  space,  an  image  on  the  image  plane. 


4 


Fig.  3  Ray  tracing  of  our  new  PAL  design 
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This  way  of  thinking  is  backed  up  by  the  findings  that  when  blocking  a  part  of  the 
PAL  not  only  the  speed  of  the  lens  decreases,  but  a  part  of  the  image  has  been  lost.  Thus 
based  on  this  idea  we  developed  an  optical  software,  one  result  of  wich  is  shown  in  the  ray 
tracing  diagram  of  Fig.3.  The  rays  originating  from  different  object  points  are  displayed  on 
the  image  plane  by  different  sections  of  the  optical  surface  in  the  sequence  1 -2-3-4.  The 
software  takes  this  into  account  automatically.  Please  note  that  from  manufactoring  point  of 
view  these  surfaces  are  well  defined. 

Another  pleasant  feature  of  this  design  is  that  the  virtual  ringshaped  panoramic  image 
shows  up  outside  the  glass  block,  thus,  it  is  suitable  for  designing  a  foveal  lens  system  which, 
when  properly  coupled  to  the  PAL,  renders  the  same  image  quality  as  the  PAL  itself. 

Choosing  a  glass  material  with  refractive  index  1.80518  we  have  achieved  a  shape  that 
allowed  us  to  optimize  the  upper  viewing  angle  to  41°  and  the  lower  viewing  angle  to  39°. 
This  means  that  the  total  viewing  angle  in  this  case  is  80°.  Fig.  2  shows  the  ray  tracing  of  this 
design.  One  can  clearly  see  that  the  first  reflecting  surface  is  defined  by  two  different  radii, 
(2,4)  instead  of  just  one. 

In  the  past  PALs  having  a  diameter  of  about  4o  mm  seemed  to  be  adaquate  for  most 
task,  but  now  we  choose  25  mm  to  decrease  weight  and  to  have  better  exploitation  of  light. 

Foveal  lens  with  viewing  angle  of  about  20° 


The  first  problem  in 
designing  a  foveal  lens,  -  i.e.,  a  lens 
which  conveys  the  optical 
information  collected  in  cartesian 
coordinates  from  a  viewing  angle  of 
about  20°,  -  is  that  the  image  has  to 
fill  out  that  part  of  the  ringshaped 
image  where  otherwise  no  optical 
information  is  present.  (In  ordinary 
PAL-images,  namely,  one  can  see  in 
the  center  only  a  black  spot.) 


F-Jliu'1'**  IFL-ii+as 


I  ,,jph"Kitl  *tri  ml  ■ 1.JW 
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Fig. 4..  Ray  tracing  of  the  foveal  lens 


Fig.  5.  Ray  tracing  in  Humanoid  PAL  (HPAL) 
vision  module 


The  second  problem  arises  from  the  fact  that 
if  the  image  plane  of  the  foveal  lens  and  the  virtual 
image  plane  of  the  PAL  do  not  coincide,  one  or  both 
of  the  displayed  images  will  be  out  of  focus.  The 
ray  tracing  of  Fig.4.  clearly  shows  that  we  succeded 
to  overcome  this  difficulty  in  our  present 
development. 

The  ray  tracing  of  Fig.  5. shows  how  this  idea 
was  realized  when  the  foveal  lens  was  put  in  front  of 
the  newly  designed  PAL,  so  that  their  optical  axes 
coincided. 
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Placing  a  rotatable  and  inclinable  mirror  in  the  optical  axis  of  the  HPAL  a  vision  sys¬ 
tem  can  be  designed,  which  is  capable  of  displaying  any  predetermined  portion  of  the  360° 
ringshaped  panoramic  image  of  the  3D  space  surrounding  the  vision  module,  in  that  part  of 
the  ringshaped  image  where  otherwise  no  optical  information  is  present. 


Fig.  5.  HPAL  vision  module 


This  means  that  such  a  vision  module  will  act  similarly  to  the  human  vision  (this  is 
the  reason  why  it  is  referred  to  as  Humanoid  PAL ,  HPAL)  since  it  extracts  the  information 
really  relevant  to  the  given  task. 


Technical  realization  of  the  designs 


When  starting  the  technical  realization  of  the  PAL  with  increased  viewing  angle,  we 
were  facing  two  problems  compared  to  the  PALs  where  the  virtual  ringshaped  image  was  in¬ 
side  of  the  glass  block.  One,  the  less  severe,  was  connected  to  the  grinding  procedure  of  the 
optic,  namely,  that  the  first  reflecting  surface  had  two  different  radii.  The  second  one  issued 
from  the  fact  that  some  portions  of  the  concave  surfaces  had  to  be  coated  with  high  reflective 
mirroring  layer,  while  other  portions  with  broad  band  antireflective  layers.  Further,  an  aper¬ 
ture  allowing  the  appropriate  image 
forming  of  the  foveal  lens  was  required. 

The  tool  that  makes  it  possible  to 
perform  the  above-mentioned  procedures  1 
was  designed  with  the  intention  that  it 
could  be  used  again  and  again.  It  , 
consisted  of  three  main  parts.  The  sketch 
of  Fig.  6  shows  that  part  of  the  tool  that 
is  used  during  antireflection  coating. 

Fig.  6.  Position  of  PAL  during  antireflective  coating 
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In  Fig.  7  one  can  see  how  the  PAL  is  inserted  into  that  part  of  the  tool  that  is  used 
during  mirroring  coating  of  the  first  reflecting  surface,  while  Fig.  8  shows  that  part  of  the  tool 
that  is  used  when  the  aperture  for  the  foveal  lens  is  being  produced  during  coating. 


i 


Fig.  7  Position  of  PAL  during  mirroring  coating 


Fig.  8  Position  of  PAL  forming  the  foveal  aperture 


Fig.  9  Technical  realization  of  the  tools  used  in  the  coating  procedures 


Fig.  10  Result  of  the  coating  procedure:  HPAL.  Observe  the  foveal  aperture  in  the  center  of  the  concave  surface 


Motor  for  rotating  the  foveal  mirror 

We  have  set  the  requirements  for  the  motor  to  be  used  for  rotating  the  foveal  mirror 
that  can  be  tilted  at  will  as  follows: 
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1.  It  should  be  simple  and  lightweight  structure  with  a  diameter  of  less  than  40  mm. 

2.  It  should  allow  direct  drive. 

3.  It  should  not  require  an  electromagnetic  brake. 

4.  Speed  control  should  be  possible  in  stepless  regulation. 

5.  I  should  allow  high  precision  speed  control  and  position  control. 

Thus,  we  have  selected  an  ultrasonic  motor,  a  motor  based  on  a  new  concept,  which 
does  not  use  magnetic  force  as  its  driving  source.  With  other  words,  it  does  not  use  coils  of 
magnets.  They  use  ultrasonic  vibration  (20  kHz  or  above)  as  their  driving  source.  They  are 
built  up  of  a  stator  made  of  piezoelectric  ceramic,  and  a  rotor  that  are  pressed  against  each 
other  with  strong  pressure  to  create  tight  adhesion.  A  progressive  ultrasonic  wave  travels 
along  the  surface  of  the  stator  while  undulating  (Fig.  11).  Only  the  vertexes  of  the  progres¬ 
sive  wave  contact  the  rotor  surface,  and,  as  a  result,  an  elliptical  rotary  motion  is  generated  at 
each  vertex.  Since  comb-tooth-shaped  groves  are  on  the  stator  surface  that  adheres  to  the 
motor,  the  shaft  of  the  motor  will  rotate  clockwise  or  counterclockwise,  depending  upon  the 
direction  of  the  progressive  wave's  travelling  (Fig.  12. 


Fig.  1 1  Inside  view  of  a  US  motor  Fig.  12  Explanatory  diagram  of  functioning 

For  rotating  the  tiltable  mirror  in  the  humanoid  vision  system  we  have  chosen  an  ultra¬ 
sonic  motor  Type  USR30-B3  from  Shinsei  Corporation,  which  is  coupled  to  an  encoder.  (Fig. 

13)  It  has  also  a  factory-adjusted  driver  with  dimensions  of  71  by  61  by  22  millimeters.  (Fig. 

14)  The  total  weight  of  the  system  is  only  125  gram;  therefore,  it  seems  to  be  rather  suitable 
for  HPAL-based  portative  instruments. 


Fig.  13  USR30-B3  (036  mm)  motor  with  encoder  Fig.  14  Factory-adjusted  US  motor  drive 
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The  mount  system  of  HPAL 

The  mount  consisting  of  several  parts  was  deigned  in  such  a  way  that  off-the-shelf 
lens  systems  can  be  used  as  relay  lens.  So,  e.g.,  the  finite  conjugate  micro  video  imaging 
lenses  of  EDMUNDS  Industrial  Optics. 

This  is  achieved  by  having  three  positioning  options: 

a)  The  optical  distances  between  the  HPAL  and  the  relay  lens, 

b)  between  the  relay  lens  and  the  camera  mount, 

c)  between  the  camera  mount  and  the  CCD  target, 
can  be  set  independently. 

Fig.  15  shows  the  various  parts  of  the  mount  system. 


Fig.  15  Various  parts  of  the  mount  system 
Please  note  that  the  foveal  lens  is  integrated  with  HPAL  into  one  single  unit 


We  integrated  the  HPAL,  the  mirror  and  the  US -motor  via  a  glass  cylinder  into  a  single  vision  module 
suitable  for  designing  and  realizing  ADS  and  SFDS. 


Fig.  16.  Workshop  drawing  and  realization  of  HPAL  based  vision  module 
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Fig.  17. demonstrates  well  that  the 
foveal  vision  displayed  in  STW  is  about  3.5 
times  larger  than  that  parts  of  the  panoramic 
peripheral  view  which  it  represents. 

Theoretically  it  is  possible  to  design  a 
zoom  foveal  lens  and  our  new  optical  design 
program  is  capable  to  this. 

Naturally  the  manufacturing  of  a 
zoom  foveal  lens  is  rather  expensive. 
Therefore  we  designed  a  fix  focus  foveal  lens 
the  viewing  angel  of  which  well  corresponds 
with  the  viewing  angel  not  covered  by  PAL. 
(Video  demonstration ) 


Fig.  17.  Enlarged  foveal  view  of  a  selected  peripheral  site. 


Software  SATELLITE  III 

During  our  collaboration  with  AnaLogic  Computers  Ltd.  it  has  turned  out  that  the 
ACE4k  chip  available  was  not  only  of  very  low  resolution  but  it  was  not  really  designed  to 
have  dedicated  direct  optical  input.  Thus,  the  light  sensitivity  of  the  chip  was  quite  low,  and 
to  eliminate  the  consequence  of  noise  effects  rather  sophisticated  methods  had  to  be  used  to 
filter  out  regular  and  stochastic  errors.  This  experience  has  only  strengthened  our  previously 
emphasized  standpoint,  namely,  that  one  should  back  up  the  ACE4k  chip  with  another  chip. 

Our  first  choice  for  the  secondary  chip  was  a  space-variant  CMOS  retina-like  digital 
chip  used  in  the  digital  camera  known  as  GIOTTO,  developed  by  Prof.  Guilio  Sandini  of  the 
University  of  Genova,  Italy.  It  would  have  been  ideal  for  PAL,  since  it  has  an  increasing 
resolution  of  the  periphery  up  to  the  center;  further,  it  allows  real-time  switching  from  log 
polar  imaging  to  STW  imaging. 

Although  it  is  not  on  the  market  yet,  we  had  the  promise  we  could  buy  one  from  the 
University.  Unfortunately,  this  could  not  be  realized,  therefore,  we  decided  to  continue  our 
investigations  with  a  commercially  available  color  camera  that  uses  a  Vi"  chip  at  least. 

We  thought  it  was  necessary  to  improve  the  image  processing  capability  of  software  programs 
SATELLITE  I  and  II,  respectively,  we  reported  on  previously.  To  be  able  to  operate  these  in 
quasi  real-time,  (about  15  images  per  second),  we  developed  SATELLITE  III,  which  is  capa¬ 
ble  to  unwrap  the  panoramic  ringshaped  image  continuously.  It  can  also  perform  various  fil¬ 
tering  procedures,  fine  edge  detection,  major  edge  extraction,  summation,  subtraction,  etc.; 
further;  it  can  provide  a  histogram  from  any  direction  of  the  foveal  view. 

On  Fig.  18  on  can  see  the  monitor  screen  showing  various  options  for  processing  the 
PAL-image  display.  The  first  icon  button  on  the  left  of  the  top  row  of  the  Dialog  Box  indi¬ 
cates  whether  or  not  the  camera  equipped  with  the  PAL  is  connected  and  ready  for  process¬ 
ing.  The  next  four  buttons  serve  to  calibrate  (set  angles,  upper  angle  position,  lower  angle  po¬ 
sition,  center  point  position).  If  once  calibrated,  there  is  no  need  to  re-calibrate  the  system 
again  and  again,  except  only  after  the  camera  and/or  the  PAL  was  changed.  The  next  button 
with  inscription  AVI  can  be  used  to  record  real-time  happenings  the  length  of  which  is  limited 
only  by  the  capacity  of  the  computer  memory.  The  remaining  buttons  activate  various  filter¬ 
ing  operations.  ( Video  demonstration) 
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Fig.  18. Dialog  box  of  software  SATELLITE  III  as  displayed  on  the  monitor  screen 


Clicking  on  the  key  Connection  one  connects  the  CCD  camera,  which  can  be  picked 
out  from  more  than  50  brands,  to  the  computer.  We  used  in  our  experiments  a  PAL  CCIR 
(Fig.  19) 


Fig.  19  Menu  of  selectable  CCD  cameras 

Clicking  on  the  key  Pan  start,  the  unwrapped  version  of  the  PAL  image  will  be  dis¬ 
played  on  Panel2  in  real  time.  (Fig.  20)  Activating  the  key  Stop,  and  clicking  on  the  key 
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Split  Start,  -  after  choosing  the  direction  East  (E),  North  (N),  West  (W)  or  South  (S)  -  the 
unwrapped  image  will  appear  on  Panel2  and  Panel3,  each  covering  a  viewing  angle  of  180°. 
(Fig.  21) 


Fig.  20.  Unwrapped  version  of  a  PAL  image.  Blue  line  indicates  the  OH  of  the  PAL 


Fig.  21. Peripheral  view  in  the  direction  of  flight  (left),  and  backwards  (right)  at  a  given  moment 


Azimuth  Determination  System  (ADS)  & 

Sniper  Fire  Location  Display  (SFLD) 

From  the  point  of  view  of  real  time  image  processing  there  is  practically  no  difference 
between  the  software  to  be  used  in  ADS  and/or  SFLD.  In  both  cases  the  well  known  charac¬ 
teristics  of  PAL  is  exploited,  namely  that  the  annular  image  represents  the  2D  skeleton  of  the 
3D  surrounding,  and  is  a  function  of  the  viewing  angle  a  of  the  PAL,  as  well  as  of  HPAL.  It  is 
a  function  of  the  viewing  angle  of  the  Pal  that  can  be  described  as  the  sum  of  the  viewing  an¬ 
gles  below  (a0  and  above  (a2)  the  plane  of  the  horizon  of  the  optic. 

When  the  panoramic  annular  image  formed  by  the  optic  is  projected  onto  the  mosaic 
target  of  a  chip,  each  pixel  along  the  radii  of  the  projected  annular  image  displays  not  only 
intensity  values  but  also  a  vector  value.  The  reason  for  this  is  that  radial  lines  mean  lines  in 
space  parallel  to  the  axis  of  PAL,  which,  however,  is  always  perpendicular  to  the  horizon 
plane  of  the  optic. 

Thus,  if  an  object  point  is  at  a  distance  R  from  the  optical  axis  and  at  a  distance  hi  be¬ 
low  the  plane  of  the  horizon  one  can  write: 

R  =  hi/tg  aj 

If  it  is  above  the  plane  of  the  horizon,  at  a  distance  It: 

R  =  tfe/tg  a2 

If  now  the  distance  (height)  H  of  the  plane  of  the  optical  horizon  from  a  well  defined 
location  is  known,  and  its  inclination  from  the  actual  horizon  d  is  also  given,  the  absolute 
distance  of  the  object  point  (target)  from  the  PAL-based  vision  module  can  immediately  be 
calculated.  The  value  H  can  be  obtained,  e.g.,  via  GPS.  The  flowchart  of  this  procedure  is 
shown  in  Fig.22. 

The  main  difference  between  the  vision  modules  of  ADS  and  SFLD  is  that  in  the  first 
case  only  PAL  is  used  while  SFLD  needs  HPAL,  since  its  task  is  not  only  to  detect  but  also  to 
pinpoint  the  target,  e.g.,  in  such  a  way  that  it  can  be  used  for  missile  guidance. 
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□ 


Helicopter  blade-tracking  system  (HBTS) 

In  the  Final  Report  of  the  Project  SPC-99-4065,  Contract  Order  No.  F61775-99- 
WE065, 1  indicated  that  a  PAL-based  measuring  system  could  perhaps  be  developed  for  heli¬ 
copter  blade  tracking.  The  proposed  system  is  based  on  the  fact  that  all  rotating  blades  are  of 
the  same  length,  thus,  if  they  are  balanced  correctly,  their  tips  will  run  on  the  same  circle. 
Placing  a  PAL-equipped  CCD  camera,  the  optical  axis  of  which  coincides  with  the  center  of 
this  main  rotor  drive  shaft,  on  the  top  of  the  dome  protecting  the  rotating  main  shaft,  the  PAL 
image  will  not  only  show  the  blades  being  out  of  track  or  not,  but  it  will  give  also  a  numerical 
measure  of  being  out  of  track. 

I  have  approached  the  87th  Bakony  Combat  Helicopter  Regiment's  Base  in  Szentki- 
ralyszabadja,  Hungary,  and  asked  them  to  help  me  to  check  how  this  idea  may  work  in  reality. 
As  a  first  step,  I  received  the  drawings  of  the  main  rotary  shaft  and  that  of  the  protective  dome 
(Fig.  23),  in  order  that  I  design  the  structure  holding  a  CCD  camera  equipped  with  PAL  of  a 
diameter  of  about  38  mm. 
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Fig.  23  Main  rotary  shaft  (left)  and  its  protective  dome  (right) 


The  workshop  of  the  Base  was  as  kind  as  to  build  this 
system,  named  ROTOPAL,  which  can  be  seen  in  Fig.  24. 

Unfortunately,  as  a  consequence  of  the  re-organization 
of  the  Hungarian  Air  Force,  the  experiments  had  to  be 
stopped,  the  new  staff  showed  no  interest  in  continuing  these 
investigations,  thus,  we  had  no  opportunity  to  mount 
ROTOPAL  on  a  helicopter  and  to  conduct  field  tests. 

Thus,  only  to  get  a  feeling  how  ROTOPAL  may  func¬ 
tion,  we  tested  the  working  of  the  idea  with  rotary  blades  of  a 
ceiling  fan,  instead  of  helicopter  rotary  blades.  Using  the 
software  SATELLITE  III  we  were  able  to  visualize  the  track¬ 
ing  behavior  of  the  fan's  rotating  blades.  We  have  chosen  a 
fan  with  a  diameter  of  about  1200  mm,  which  had  four 
shovel-form  blades.  Their  rotating  speed  could  be  varied  be¬ 
tween  1  to  20  rotations  per  minute. 

Fig.  25  shows  the  panoramic  image  of  the  four  blades. 

Fig.  24  ROTOPAL  mounted  on 
the  top  of  the  protective  dome 

More  informative  is  the  picture  of  Fig. 

26,  which  shows  the  unwrapped  version  of 
the  PAL  image.  The  undulation  of  the  tips  of 
the  blades  around  the  unwrapped  optical 
horizon  circle  clearly  indicates  that  the  blades 
are  out  of  track. 

Perhaps  even  more  information  can  be 
extracted  from  its  filtered  version,  shown  in 
Fig.  27. 

As  already  often  pointed  out,  the 
width  of  the  annulus  of  the  PAL  picture  rep¬ 
resents  the  viewing  angle  of  the  PAL  in  its 
optical  axis.  Thus,  each  pixel  along  a  given 
radius  of  the  panoramic  image  indicates  the 
direction  of  an  image  point  in  space.  (Video 
demonstration) 
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Fig.  26  Unwrapped  version  of  the  PAL  image  of  Fig.  22 


Fig.  27  Filtered  version  of  the  unwrapped  PAL  image  of  Fig.  23 

In  our  case,  the  positions  of  the  blade  tips  are  shown  in  a  given  moment.  Based  on  the 
above  mentioned  knowledge  provided  that  the  calibration  of  both,  that  of  the  ROTOPAL  and 
of  the  software,  was  accurate,  and  the  length  of  the  rotating  blades  is  known,  the  amplitude  of 
the  oscillation  of  the  blade  tips  can  be  calculated  and  expressed  in  degrees. 

CONCLUSIONS 

1.  A  new  humanoid  PAL  (HPAL)  has  been  designed  and  optimized  with  a  vertical 
viewing  angle  of  about  80°. 

2.  A  foveal  lens  with  a  viewing  angle  of  about  20°  has  been  designed  and  optimized. 

3.  Mount  system  for  HPAL-based  vision  module  has  been  designed. 

4.  Based  on  the  designs  1,  2,  and  3  a  working  model  of  a  HPAL  vision  module  using 
US  motor  for  mirror  rotating  has  been  constructed  for  field-testing. 

5.  The  software  SATELLITE- III  for  real-time  image  processing  has  been  developed 
and  tested  in  part  during  helicopter  flight. 

6.  A  satisfying  cooperation  with  AnaLogic  Computers  Ltd.  has  been  worked  out. 

Suggestions  on  how  to  proceed  to  exploit  the  presented  results 

1.  Since  a  working  vision  module  of  HPAL  using  US  motor  for  mirror  rotating  is  now 
available,  it  should  be  combined  with  GPS  and  directly  used  for  military  applications  such  as 
tracking  and  pinpointing  missiles,  independently  from  which  points  of  the  compass  they  are 
coming  from. 

2.  Since  our  simple  experiments  simulating  rotor  blade  tracking  with  a  practically 
ready-to-mount  ROTOPAL  were  very  encouraging,  and  having  a  working  vision  module  of 
HPAL  I  suggest  that  a  system  should  be  worked  out  which  is  capable  of  blade  tracking  during 
flight  maneuvers. 

3.  In  the  preliminary  design  concepts  of  Reports  of  the  Project  SPC-99-4065,  Contract 
Order  No.  F61775-99-WE065,  I  already  emphasized  that  PAL  imaging  considers  its  sur¬ 
roundings  to  be  cylindrical,  then,  if  one  places  a  mirror  cylinder  vertically  on  the  center  of  the 
annular  picture  delivered  by  the  PAL,  and  looking  at  the  cylinder  one  can  not  only  see  the 
panoramic  image  rectified  but  one  can  also  walk  around. 

This  gives  me  the  idea  to  propose  a  360°  panoramic  projection  system  for  training  of 
helicopter  pilot  personnel,  based  on  those  PAL  video  recordings,  which  were  shot  during 
flight  with  a  vision  module  based  on  HPAL. 

We  tested  this  idea  by  using  a  monitor  having  its  screen  horizontally  and  projecting  on 
it  in  real  time  the  pictures  recorded  by  a  PAL  vision  module. 

Placing  a  mirror  cylinder  in  the  center  of  the  projected  image,  which  is  usually  black, 
one  can  see  in  the  rectified  panoramic  image  all  movements  that  happened  during  recording. 
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Such  a  pilot  system  can  be  seen  in  Fig.  28. 


Fig.  28  Ring-shaped  PAL  image  that  was  recorded  with  a  PAL-equipped  video  camera  and  is  displayed  on  a  flat 
monitor  lying  horizontally  shows  the  happenings  in  the  surrounding  space  and  is  simultaneously  being  “recti¬ 
fied”  with  a  mirror  cylinder  placed  in  the  central  black  spot. 

A  working  model  of  such  a  system,  called  anamorphic  panoramic  video,  can  be  seen 
till  the  end  of  the  year  2002  at  the  one-year  exhibition  of  the  Millennium  Park,  Budapest. 
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